Comments for MEDB 5501, Week 6

Assessing normality

Problems caused by non-normality
- Poor confidence intervals, hypothesis tests
  - Too much imprecision
  - Poor coverage probability
    - Especially for one tailed tests
- Inability to extrapolate
What about the Central Limit Theorem?

Sometimes, I think that researchers obsess too much about non-normality, but in fairness to them, it is an important issue. If your data does not follow a bell shaped curve, then several problems could happen.

First, you might see a greater degree of imprecision, reflected in very wide confidence intervals and loss of statistical power.

Second, you might have poor coverage probability. The 95% confidence interval might only reprent a 92% confidence level. The Type I error rate might be greater than 5%.

Third, and this point is not emphasized enough, is that non-normal data makes it difficult for you to extrapolate to future observations. Prediction of future events is a big part of Statistics. It is difficult even with normal data and becomes much more difficult with non-normal data.

Yes, you might say, but doesn’t the Central Limit Theorem help us out? Well, yes, if the sample size is large, but it only helps with assuring accurate confidence levels and good control of the Type I error rate. Non-normal data will still often produce intervals that are too wide and tests that have too little power.

How to handle non-normality

Ignore it
- Central Limit Theorem
Transform your data
Use alternatives
- Nonparametric tests (covered in a later module)
- Bootstrap (covered in a later module)
- Randomization tests (not covered in this class)

Yes, you might say, but doesn’t the Central Limit Theorem help us out? Well, yes, but it only helps with assuring accurate confidence levels and good control of the Type I error rate. Non-normal data will still often produce intervals that are too wide and tests that have too little power.

Still, there often is benefit in not worrying so much about non-normality. If your sample size is large and the deviations from normality are minor, don’t let this keep you up at night.

Even so, there are some benefits to addressing non-normality. Transformations, which I will cover in just a bit, can often help out tremendously. There are also alternatives to the simple tests using a sample mean: nonparametric tests and the bootstrap. I will cover those in a later module. There are also randomization tests, which I don’t think are covered. I’ve given a talk on randomization tests and I have a few webpages that talk about this topic.

Normal histogram with n=100

Normal histogram with n=1,000

Normal histogram with n=10,000

Normal histogram with n=100,000

Normal histogram with n=1,000,000

Normal distribution (n=infinity)

Normal(2, 1)

Normal(-1, 1)

Normal(0, 2)

Normal(0, 0.5)

Skewed right

Here’s an example of a distribution that is skewed to the right. I have scaled this distribution so that the mean is zero and the standard deviation is one.

There are three key features for a skewed right distribution.

First, the chances of seeing an outlier on the low end (more than two standard deviations below the mean) is less for a skewed right distribution. Skewed right distributions almost never produce outliers on the low end.

Second, the chances of seeing an outlier on the high end (more than two standard deviations above the mean) is greater for a skewed right distribution.

Third, the probability of any value being less than the mean is a bit more than 50%. That’s because the outliers on the high end of this distribution tend to have an undue influence on the mean, making it bigger than the bulk of the data.

Right skewness is characterized by the tails of the distribution

Heavy right tail
- Greater tendency to produce extreme values on the right
Light left tail
- Lesser tendency to produce extreme values on the left
Right skewness is the most common type of non-normality

Normal probability plot

Compare data to evenly spaced percentiles of the normal distribution
Example with n=4
- Compare smallest value with \(Z_{0.2}\)
- Compare next value with \(Z_{0.4}\)
- Compare next value with \(Z_{0.6}\)
- Compare largest value with \(Z_{0.8}\)
No best definition for evenly spaced
- 12.5, 37.5, 62.5, 87.5, for example